Communications  Research  for  Command  &  Control:  Human-Machine  Interface 
Technologies  Supporting  Effective  Air  Battle  Management 


Robert  S.  Bolia 

Air  Force  Research  Laboratory  (AFRL/HECP) 
2255  H  Street 

Wright-Patterson  Air  Force  Base,  OH  45433-7022 
W.  Todd  Nelson 

Air  Force  Research  Laboratory  (AFRL/HECP) 
2255  H  Street 

Wright-Patterson  Air  Force  Base,  OH  45433-7022 
Michael  A.  Vidulich 

Air  Force  Research  Laboratory  (AFRL/HECP) 
2255  H  Street 

Wright-Patterson  Air  Force  Base,  OH  45433-7022 
Brian  D.  Simpson 

Air  Force  Research  Laboratory  (AFRL/HECB) 
2610  Seventh  Street 

Wright-Patterson  Air  Force  Base,  OH  45433 
Douglas  S.  Brungart 

Air  Force  Research  Laboratory  (AFRL/HECB) 
2610  Seventh  Street 

Wright-Patterson  Air  Force  Base,  OH  45433 


Report  Documentation  Page 

Form  Approved 

OMB  No.  0704-0188 

Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 

VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  OMB  control  number. 

1.  REPORT  DATE 

JUN  2005  2.  REPORT  TYPE 

3.  DATES  COVERED 

00-00-2005  to  00-00-2005 

4.  TITLE  AND  SUBTITLE 

Communications  Research  fro  Command  &  Control:  Human-Machine 
Interface  Technologies  Supporting  Effective  Air  Battle  Management 

5a.  CONTRACT  NUMBER 

5b.  GRANT  NUMBER 

5c.  PROGRAM  ELEMENT  NUMBER 

6.  AUTHOR(S) 

5d.  PROJECT  NUMBER 

5e.  TASK  NUMBER 

5f.  WORK  UNIT  NUMBER 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

Air  Force  Research  Laboratory ,AFRL/HECP, 2255  H  Street, Wright 
Patterson  AFB, OH, 45433-7022 

8.  PERFORMING  ORGANIZATION 

REPORT  NUMBER 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 

10.  SPONSOR/MONITOR'S  ACRONYM(S) 

11.  SPONSOR/MONITOR'S  REPORT 
NUMBER(S) 

12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Approved  for  public  release;  distribution  unlimited 

13.  SUPPLEMENTARY  NOTES 

The  original  document  contains  color  images. 

14.  ABSTRACT 

15.  SUBJECT  TERMS 

16.  SECURITY  CLASSIFICATION  OF:  17.  LIMITATION  OF 

18.  NUMBER  19a.  NAME  OF 

a.  REPORT  b.  ABSTRACT  c.  THIS  PAGE 

unclassified  unclassified  unclassified 

40 

Standard  Form  298  (Rev.  8-98) 

Prescribed  by  ANSI  Std  Z39-18 


Communications  Research  for  Command  &  Control:  Human-Machine  Interface 
Technologies  Supporting  Effective  Air  Battle  Management 


Robert  S.  Bolia 
W.  Todd  Nelson 
Michael  A.  Vidulich 
Brian  D.  Simpson 
Douglas  S.  Brungart 
Air  Force  Research  Laboratory 
Wright-Patterson  Air  Force  Base,  OFl 


Abstract.  Battle  management  command  and  control  (BMC2)  is  a  communications- 
intensive  activity.  Weapons  directors  and  mission  crew  commanders  on  these  platforms 
are  required  to  monitor  as  many  as  eight  simultaneous  communications  channels  against 
a  background  of  moderate  to  high  ambient  cabin  noise  while  perfonning  a  number  of 
visual  and  manual  tasks,  a  combination  which  in  the  heat  of  battle  is  challenging  even  for 
the  most  highly  trained  operators.  Researchers  at  the  Air  Force  Research  Laboratory’s 
Human  Effectiveness  Directorate  (AFRL/HE)  have  been  investigating  two  technologies 
to  ameliorate  this  problem:  active  noise  reduction  (ANR)  headsets  and  spatial  intercoms. 
ANR  headsets  cancel  environmental  noise  while  preserving  speech  signals  presented  via 
the  communications  network,  while  spatial  intercoms  enhance  communications  by 
increasing  the  effective  signal-to-noise  ratio  of  each  individual  communications  channel, 
taking  advantage  of  the  listener’s  natural  ability  to  efficiently  segregate  speech  streams 
that  are  separated  in  space.  This  paper  will  describe  investigations  at  AFRL/HE  directed 
at  the  enhancement  of  speech  intelligibility  in  multi-channel  tactical  BMC2  environments 
using  ANR  and  spatial  intercom  technology.  Results  will  be  discussed  from  basic 
laboratory  experimentation,  simulated  mission  scenarios,  and  field  evaluations,  and 
interpreted  in  the  context  of  the  acquisition  and  integration  of  such  technologies  for 
current  and  future  BMC2  weapons  systems. 


Introduction.  Battle  management  command  and  control  (BMC2)  as  performed  by 
operators  of  tactical  air  battle  management  platforms  such  as  the  US  Air  Force’s  E-3 
Airborne  Warning  and  Control  System  (AWACS)  or  E-8  Joint  Surveillance  Target 
Attack  Radar  System  (JSTARS),  or  the  US  Navy’s  E-2C  Hawkeye,  is  a  communications- 
intensive  activity.  Weapons  directors  and  mission  crew  commanders  on  these  platforms 
are  required  to  monitor  as  many  as  eight  simultaneous  communications  channels  against 
a  background  of  moderate  to  high  ambient  cabin  noise  while  perfonning  a  number  of 
visual  and  manual  tasks,  the  combination  of  which  in  the  heat  of  battle  is  challenging 
even  for  the  most  highly  trained  operators.  Researchers  at  the  Air  Force  Research 
Laboratory’s  Human  Effectiveness  Directorate  have  been  investigating  two  technologies 
that  may  ameliorate  this  problem:  active  noise  reduction  headsets  and  spatial  intercoms. 

Active  noise  reduction  (ANR)  headsets  make  use  of  the  phenomenon  of  destructive 
interference  to  cancel  low-frequency  environmental  noise  while  preserving  the  speech 
signals  presented  to  operators  via  the  communications  network.  This  is  accomplished  by 


measuring  the  external  noise  field  at  each  ear,  inverting  the  phase  of  the  waveform,  and 
adding  it  to  the  original  waveform  by  means  of  a  speaker  placed  inside  the  ear  cup.  In 
addition  to  improving  communications  effectiveness,  ANR  can  reduce  the  hearing  loss 
attendant  from  long-duration  sorties  and  the  practice  of  increasing  the  volume  of  a 
particular  net  or  radio  channel  to  a  dangerous  level  in  order  that  it  might  be  audible  above 
the  compound  background  of  noise  and  other  speech. 

Spatial  intercoms  (Bolia  &  Nelson,  2003)  endeavor  to  enhance  communications  by 
increasing  the  effective  signal-to-noise  ratio  of  any  particular  communications  channel, 
taking  advantage  of  the  human  listener’s  natural  ability  to  more  efficiently  segregate 
speech  streams  that  are  separated  in  space.  This  is  accomplished  by  digitally  filtering 
discrete  channels  using  transformations  derived  from  the  physical  cues  used  for  sound 
localization  to  make  the  speech  on  each  channel  appear  to  emanate  from  distinct 
locations  in  space.  Research  has  demonstrated  that  such  apparent  separation  can  lead  to 
substantial  improvements  in  speech  intelligibility  and  reductions  in  communications 
workload. 

This  paper  will  describe  investigations  at  the  Air  Force  Research  Laboratory 
directed  at  the  enhancement  of  speech  intelligibility  in  multi-channel  tactical  BMC2 
environments  using  ANR  and  spatial  intercom  technology.  Results  will  be  discussed 
from  basic  laboratory  experimentation,  simulated  mission  scenarios,  and  field  evaluations, 
and  interpreted  in  the  context  of  the  acquisition  and  integration  of  such  technologies  for 
current  and  future  BMC2  weapons  systems. 


Active  Noise  Reduction  Headsets.  The  moderate  to  high  ambient  cabin  noise  present  on 
airborne  air  battle  management  platfonns  such  as  AWACS  and  JSTARS  and  the  duration 
of  missions  flown  point  to  the  potential  for  hearing  loss  in  both  the  flight  and  the  mission 
crews  (Mobley,  Hall,  &  Yeager,  2002).  This  danger  is  exacerbated  by  the  tendency  of 
some  crewmembers  to  wear  their  headsets  with  one  ear  exposed  in  order  to  facilitate 
communication  with  other  crewmembers  without  squandering  the  onboard  intercoms. 
This  activity  may  lead  to  asymmetric  hearing  loss,  further  impairing  communication 
effectiveness. 

In  order  to  improve  communication  effectiveness  and  reduce  the  potential  for 
hearing  loss,  the  AWACS  System  Program  Office  (SPO)  decided  to  purchase  ANR 
headsets  to  replace  the  current  headset  in  use  by  the  AWACS  crews.  Inasmuch  as  several 
commercial  producers  could  provide  ANR  headsets,  the  AWACS  SPO  requested  that 
AFRL  conduct  an  evaluation  to  determine  the  best  overall  choice  for  use  in  the  AWACS 
environment  (Vidulich,  Donnelly,  McLaughlin,  Read,  Hall,  Schley,  Mobley,  Bolia, 
Nelson,  &  Poole,  2003). 

The  evaluation  included  several  objectively  measured  headset  attributes  as  well  as 
subjective  ratings.  The  list  of  objective  criteria  included:  Noise  Attenuation 
Performance,  Headset  Clamp  Strength,  Headset  Weight,  Battery  Pack  Weight,  and  the 
Type  and  Number  of  Batteries  Required.  The  values  for  all  of  these  were  detennined  by 
measurement  or  observation  in  AFRL  laboratories.  For  the  purposes  of  comparison  the 
actual  physical  measurements  on  the  various  objective  parameters  were  not  used  in  the 
later  calculations.  For  any  measurement  class,  the  raw  values  were  converted  to  a 
relative  value  scale  that  identified  how  each  headset  compared  to  the  “average”  headset. 


The  subjective  evaluation  criteria  were  any  opinion-based  values  derived  from  the 
ratings  of  the  operators  that  tested  the  headsets  in  representative  flight  operations.  The 
evaluation  team  defined  five  subjective  dimensions  to  be  assessed  by  the  ANR  headset 
evaluators:  Fit,  Comfort,  Usability,  Audio  Quality,  and  Hearing  Protection  &  Fatigue. 
The  subjective  ratings  were  collected  from  AWACS  aircrews  perfonning  nonnal  aircrew 
duties  during  several  combat  training  sorties  flown  out  of  Tinker  Air  Force  Base, 
Oklahoma.  The  fact  that  different  headsets  were  assessed  by  different  teams  of  operators 
was  problematic  due  to  the  potential  influence  of  individual  differences  on  subjective 
ratings.  To  minimize  these  influences  as  much  as  possible,  a  relative  comparison 
approach  was  used  (Tsang  &  Vidulich,  1994).  As  all  of  the  operators  had  considerable 
experience  with  the  current  headset,  it  was  used  as  the  baseline  and  all  subjective 
evaluations  were  based  on  comparison  to  it.  This  approach  had  the  added  advantage  of 
directly  testing  whether  the  headsets  under  evaluation  constitute  an  improvement  over  the 
current  headset  (Rueb,  Vidulich,  &  Hassoun,  1994). 

A  decision  matrix  procedure  was  used  to  combine  the  raw  data  according  to  the 
relative  importance  weights  provided  by  subject-matter  experts  (Halpem,  2002).  Overall, 
the  results  strongly  supported  the  potential  value  of  ANR  headsets  in  the  in-flight 
environment.  ANR  headsets  were  assessed  as  improving  communications  throughput,  in 
addition  to  providing  greater  hearing  protection. 

Based  on  the  parameters  considered  in  the  evaluation,  the  Bose  AHX  headset  was 
the  clear  winner  over  the  other  tested  alternatives.  The  subjective  evaluations  for  the 
Bose  headset  were  strongly  positive,  the  headset  weight  and  head  clamp  strength 
measurements  were  very  good,  and  there  were  no  negative  factors  found.  The  Bose 
headset  was  also  perceived  as  having  better  than  average  attenuation  performance  among 
the  headsets  evaluated. 

As  an  extra  precaution  to  be  sure  that  the  Bose  headset  was  the  best  choice,  a 
second  evaluation  was  conducted  (McLaughlin,  Vidulich,  Donnelly,  Bolia,  &  Nelson, 
2003).  In  this  evaluation  the  Bose  headset  and  the  second-place  headset  from  the  first 
evaluation  were  subjected  to  a  longer  in-flight  usability  evaluation.  A  survey  assessment 
tool  was  used  to  collect  data  and  opinions  from  AWACS  crew  members  as  they  used  the 
headsets  for  about  four  weeks.  Overall,  the  pattern  of  results  strongly  supported  the  Bose 
headset  as  a  better  choice  for  AWACS  use.  The  ratings  and  qualitative  comments 
showed  that  the  Bose  headset  was  considered  a  substantial  improvement  relative  to  the 
current  headset,  but  that  the  second-place  headset  was  more  prone  to  comfort  and 
usability  issues. 

Based  on  these  two  evaluations,  the  Bose  headset  was  purchased  for  use  on  the 
AWACS  aircraft. 


Spatial  Intercoms.  That  spatial  intercoms  are  possible  is  due  to  a  phenomenon  known 
as  the  “cocktail  party  effect”  (Cherry,  1953),  which  refers  to  the  fact  that,  at  a  cocktail 
party,  in  the  midst  of  perhaps  scores  of  people  engaged  in  dozens  of  conversations,  an 
individual  can  still  make  sense  of  what  the  person  she  is  talking  to  is  trying  to  say,  despite 
the  fact  that  if  the  same  conversations  were  played  to  the  individual  over  mono 
headphones  she  would  be  hopelessly  lost.  While  many  factors  contribute  to  the  cocktail 


party  effect  (see  Bronkhorst,  2000;  and  Yost,  1997,  for  reviews),  one  of  the  most  often 
studied  is  the  spatial  separation  of  competing  talkers. 

It  has  long  been  known  that  listening  to  two  or  more  speakers  simultaneously  is 
easier  if  the  two  are  at  different  locations.  In  the  real  world  this  may  not  be  obvious, 
since  multiple  talkers  seldom  inhabit  exactly  the  same  spot.  However,  listening  to  even 
two  overlapping  talkers  over  headphones,  or  from  the  same  loudspeaker,  is  a  challenging 
and  often  frustrating  task. 

If  spatial  separation  enhances  the  intelligibility  of  multi-talker  conversations,  a 
display  that  is  able  to  effectively  simulate  distinct  localizable  sound  sources  might  also  be 
adapted  for  use  as  a  spatial  intercom.  The  construction  of  such  a  display  is  possible 
because  of  what  scientists  have  learned  about  the  cues  subserving  sound  localization  in 
humans,  all  of  which  are  based  on  the  interactions  of  the  incident  acoustic  wave  with  the 
head,  torso,  and  pinnae.  Two  of  these  cues,  interaural  time  differences  (ITDs)  and 
interaural  level  differences  (ILDs)  are  used  primarily  for  the  localization  of  the  sound 
source  in  azimuth.  ITDs  arise  because,  for  sound  sources  not  in  line  with  a  listener’s 
median  plane,  the  acoustic  wave  will  reach  one  ear  before  it  reaches  the  other.  Although 
these  temporal  differences  are  small  -  on  the  order  of  tens  of  microseconds  -  they  are 
readily  detectable  by  brainstem  structures.  However,  because  of  limitations  in  neural 
phase  locking,  they  are  only  useful  for  frequency  components  below  about  1500  Hz 
(Irvine,  1992).  For  frequencies  above  3000  Hz  (a  region  in  which  the  wavelength  of  an 
acoustic  stimulus  is  small  relative  to  the  size  a  listener’s  head),  the  head  casts  an  acoustic 
shadow  such  that  the  stimulus  level  at  the  contralateral  ear  (i.e.,  the  ear  on  the  side  of  the 
head  opposite  the  sound  source)  is  attenuated  relative  to  the  stimulus  level  at  the 
ipsilateral  ear.  The  resulting  ILDs  are  useful  for  localizing  sounds  in  this  frequency 
region  (Feddersen,  Sandel,  Teas,  &  Jeffress,  1957;  Blauert,  1974).  For  frequencies 
between  1500  and  3000  Hz,  little  spatial  infonnation  is  available,  and  hence  such  sounds 
are  less  localizable  by  humans.  Fortunately,  most  naturally  occurring  sounds  are 
broadband  and  include  frequency  components  in  the  regions  in  which  both  ITDs  and 
ILDs  operate. 

These  binaural  cues  are  useful  primarily  for  determining  the  azimuthal  component 
of  sounds.  However,  a  particular  set  of  binaural  cues  does  not  uniquely  specify  a  point  in 
azimuth.  Rather,  since  the  head  is  roughly  spherical,  a  sound  in  the  front  hcmi field 
would  result  in  an  ITD  and  an  ILD  identical  to  those  of  a  sound  in  the  rear  hemifield  (i.e., 
10°  has  the  same  ITD  and  ILD  as  170°,  in  a  right-handed  coordinate  system  with  0°  as 
the  position  directly  in  front  of  the  listener).  Failure  by  the  auditory  system  to 
disambiguate  the  location  of  such  sounds  can  engender  front-back  confusions  (Gardner 
and  Gardner,  1973;  Oldfield  and  Parker,  1984a;  1984b).  Both  localization  in  elevation 
(Ro filer  &  Butler,  1968)  and  the  arbitration  of  front-back  confusions  (Musicant  &  Butler, 
1984)  are  mediated  by  frequency-dependent  level  differences  occasioned  by  the 
interaction  of  the  incident  sound  wave  with  the  pinnae,  and,  to  a  lesser  extent,  the  head 
and  torso.  Unlike  ITDs  and  ILDs,  these  differences  are  highly  variable  from  listener  to 
listener,  and  the  spatial  resolution  at  which  they  can  be  discriminated  is  much  poorer  than 
that  of  either  binaural  cue. 

The  spectral  and  temporal  differences  described  above  can  be  measured  on  human 
listeners  and  captured  in  what  is  called  the  head-related  transfer  function  (HRTF; 
Wightman  &  Kistler,  1989).  If  HRTFs  are  measured  for  both  ears  for  broadband  sound 


sources  at  a  number  of  spatial  positions,  each  transfer  function  can  be  represented  as  a 
digital  filter,  the  collection  of  which  produce  a  model  of  the  listener’s  auditory  world  that 
can  be  used  to  generate  veridical  spatial  perception.  Specifically,  given  a  dense  enough 
set  of  HRTFs,  an  arbitrary  audio  signal  can  be  convolved  with  the  pair  representing  the 
transformation  from  the  free  field  to  the  left  and  right  ear  at  a  specified  position,  and  be 
played  back  over  headphones  to  convey  the  appearance  of  emanating  from  that  location 
in  space.  Martin,  McAnally,  and  Senova  (2001)  have  demonstrated  that  such  a  display 
can  produce  localization  of  “virtual”  sounds  equivalent  to  free-field  localization. 

It  is  well  established  that  headphone-based  multi-talker  communications  can  be 
substantially  improved  by  separating  talkers  in  a  spatial  audio  display.  Studies  conducted 
at  the  Air  Force  Research  Laboratory  (e.g.,  Ericson,  Brungart,  and  Simpson,  2004; 
Nelson,  Bolia,  Ericson,  &  McKinley,  1998)  indicate  that  improvements  in  intelligibility 
may  be  in  excess  of  30-40%,  depending  on  the  configuration  and  number  of  simultaneous 
talkers.  However,  little  is  known  about  how  best  to  position  these  talkers  in  order  to 
support  optimal  performance.  Previous  studies  have  typically  operated  under  the 
assumption  that  a  maximum  separation  between  talkers  leads  to  best  perfonnance.  As  a 
result,  most  displays  have  employed  configurations  in  which  all  talkers  are  equally 
spaced  along  the  plane  bisecting  the  listener’s  ears  (the  horizontal  plane),  equidistant 
from  the  center  of  the  listener’s  head,  in  the  listener’s  front  hemifield.  While  this  does 
provide  maximum  separation  between  all  talkers,  it  does  not  take  into  account  the  fact 
that  auditory  spatial  acuity  is  not  homogeneous  across  the  plane  (Mills,  1958). 
Specifically,  acuity  is  best  for  sources  presented  directly  in  front  of  a  listener,  and 
becomes  progressively  worse  as  the  source  is  moved  toward  the  periphery.  Based  on  this, 
and  recent  findings  suggesting  that  listeners  can  use  differences  in  distance  as  a  means  of 
segregating  multiple  talkers  (Brungart  &  Simpson,  2002),  researchers  at  AFRL  have 
proposed  a  new  display  configuration,  capable  of  accommodating  seven  talkers  (Brungart 
&  Simpson,  2003).  In  this  display,  the  talkers  are  more  closely  spaced  in  the  front,  and 
more  widely  toward  the  periphery.  Moreover,  both  far-field  (1  m)  and  near-field  (12  cm) 
locations  were  employed.  This  configuration  can  be  seen  in  the  top  panel  on  the  left- 
hand  side  of  Figure  1.  The  corresponding  performance  benefits  for  this  spatial 
configuration  are  shown  in  the  right-hand  panel  of  the  same  figure.  These  data,  collected 
in  the  laboratory  at  AFRL,  are  compared  to  performance  in  a  diotic  configuration,  in 
which  all  channels  were  mixed  and  presented  identically  to  both  ears,  as  shown  in  the 
bottom  panel  on  the  left-hand  side  of  Figure  1.  The  diotic  condition  is  analogous  to  the 
intercom  system  that  currently  exists  in  AWACS  and  JSTARS  airborne  platforms.  As 
can  be  seen,  performance  with  the  optimal  3D  audio  configuration  is  substantially  better 
than  the  diotic  configuration,  independent  of  the  number  of  talkers. 
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Figure  1 :  Comparison  of  overall  performance  with  spatial  audio  and  diotic  audio  displays  (shown  on  the  left 
of  the  figure)  in  a  multi-talker  listening  task  with  all  male  talkers.  The  results  (shown  on  the  right)  have  been 
plotted  as  a  function  of  the  number  of  talkers  in  the  trial.  Error  bars  represent  the  95%  confidence  intervals  for 
each  data  point. 


The  clear  communication  effectiveness  advantage  found  with  spatial  intercoms  in 
the  laboratory  suggested  the  need  to  introduce  this  technology  into  the  operational 
community  in  order  to  examine  its  potential  benefit  for  individuals  well-trained  to 
monitor  multiple  communication  channels  in  complex  environments.  Therefore, 
researchers  from  AFRL  brought  these  displays  to  operators  in  the  BMC2  community. 

The  first  tests  of  this  kind  were  conducted  with  members  of  the  AWACS 
operational  community  at  Tinker  Air  Force  Base  (Bolia,  2003).  Here,  laboratory  tasks 
conducted  at  AFRL  were  replicated  with  trained  operators  in  a  controlled  environment, 
and  later  onboard  the  aircraft  on  an  AWACS  training  mission.  The  results  were  nearly 
identical  to  those  found  in  the  laboratory,  with  operators  perfonning  30-40%  better  on  the 
communication  task  with  spatial  intercoms  than  with  the  standard  diotic  communication 
system.  Similar  results  were  found  for  members  of  the  JSTARS  community  at  Robins 
Air  Force  Base,  Georgia,  and  for  the  ground-based  C2  community  at  the  USAF  Weapons 
School  at  Nellis  Air  Force  Base,  Nevada. 

Perceived  workload  was  also  evaluated  for  AWACS  operators  using  the  NASA 
Task  Load  Index  (TLX).  The  results  indicated  that  along  all  TLX  subscales  ( mental 
demand,  physical  demand,  and  temporal  demand,  performance,  frustration,  and  effort), 
workload  was  found  to  be  lower  with  spatial  audio  than  with  standard  diotic  displays 
(Bolia,  2003). 

All  of  the  studies  mentioned  above  measured  the  affects  of  spatial  audio  on 
speech  intelligibility  in  relatively  context-free  environments.  A  logical  next  step  was  the 


evaluation  of  spatial  intercoms  in  a  simulated  mission  environment.  With  this  in  mind, 
air  weapons  officers  from  Tinker  Air  Force  Base  were  brought  to  AFRL  to  control 
simulated  battlefield  air  interdiction  missions  with  and  without  spatial  comm.  Results 
indicated  that  speech  intelligibility  degraded  significantly  during  the  most  difficult 
mission  segments,  and  that  spatial  audio  alleviated  the  degree  of  this  degradation.  While 
perceived  mental  workload  was  relatively  unaffected  by  the  intercom  manipulation, 
operators  indicated  on  post-hoc  questionnaires  that  they  believed  spatial  audio  would  be 
valuable  for  improving  communications  effectiveness  (Nelson  &  Bolia,  2003). 

On  the  basis  of  these  field  and  laboratory  studies  verifying  the  operational  utility 
of  spatial  audio  for  air  battle  management,  AFRL  was  able  to  obtain  funding  to 
implement  the  technology  in  two  ground-based  controller  stations  as  part  of  a 
communications  system  upgrade  at  the  USAF  Weapons  School.  This  system,  which 
allows  users  to  allocate  incoming  voice  communications  to  seven  different  apparent 
locations,  has  made  it  possible  for  dozens  of  operators  to  experience  the  advantages  of 
spatial  audio  during  the  conduct  of  realistic  training  missions.  Operator  feedback  about 
this  system  has  been  overwhelmingly  positive: 

“With  the  use  of  the  volume  controls  and  the  various  bucket  positions  I  had  total  SA 
on  the  fight.  I  could  tell  the  difference  between  my  aircraft  and  the  adversaries  and 
was  fully  aware  when  the  pilots  were  talking  on  the  common  safety  frequency.  This  is 
the  first  time  in  my  16  year  career  that  I  have  fully  heard  and  comprehended  a  multi- 
frequency  mission.  ” 

“Increases  SA  as  to  what  radio  (frequency)  someone  is  talking  on.  Cuts  down  on 
guessing  what  radio  a  TX  came  across.  ” 

“I  think  the  system/capability  is  very  useful  for  certain  missions.  As  an  SD  listening 
to  5  or  6  differen  t  frequencies,  it  makes  a  big  difference  helping  you  allocate  SA  to 
listen  to  a  particular  freq.  ” 

These  comments  indicate  not  only  overwhelming  acceptance  by  the  user 
community,  but  also  the  increase  in  mission  effectiveness  that  is  possible  due  to  the 
enhanced  situation  awareness  (SA)  supported  by  spatial  audio  (see  also  Simpson,  Bolia, 
&  Draper,  2003;  and  Simpson,  Bolia,  &  Brungart,  in  press).  Operators  were  able  to 
maintain  an  awareness  of  the  multiple  simultaneous  communication  channels  that  were 
presented  to  them,  and  could  determine  who  was  saying  what  and  on  what  channel.  It  is 
clear  from  these  results  and  comments  that  having  a  spatial  audio  display  in  an 
operational  environment  can  provide  unambiguous  and  effective  communication,  and  a 
clear  advantage  in  the  operational  battlespace. 

One  concern  with  introducing  spatial  intercoms  into  airborne  BMC2  platforms  is 
that  the  binaural  cues  used  to  segregate  speech  channels  might  be  disrupted  by  the 
tendency  of  personnel  to  operate  with  one  ear  open.  In  order  to  test  this  hypothesis, 
researchers  at  AFRL  conducted  an  experiment  looking  at  speech  intelligibility  as  a 
function  of  the  number  and  configuration  of  simultaneous  talkers  under  both  earcup 
conditions  (both  ears  covered,  one  ear  uncovered).  The  results  of  this  study  were 
surprising,  demonstrating  that  although  perfonnance  with  a  spatial  intercom  was  reduced 


when  one  ear  was  occluded,  it  was  never  reduced  below  the  level  of  the  current  diotic 
communications  system.  Moreover,  there  were  some  configurations  for  which 
performance  using  the  spatial  configuration  was  significantly  enhanced  relative  to  the 
non-spatial  control,  even  in  the  absence  of  binaural  cues  (Bolia,  2002).  Thus,  although 
such  an  earcup  configuration  is  not  recommended  due  to  its  potential  for  inducing 
asymmetric  hearing  loss,  it  should  not  be  used  as  an  argument  against  the  introduction  of 
spatial  audio  technology. 


Spatial  Intercoms  and  ANR.  Given  the  capacity  of  both  spatial  audio  and  ANR  to 
enhance  communication  effectiveness,  it  makes  sense  to  inquire  as  to  whether  there 
might  be  a  facilitative  interaction  between  the  two.  This  would  be  of  particular  interest, 
given  that  effects  of  ANR  on  speech  intelligibility  have  seldom  been  demonstrated  (but 
see  Pellieux,  Sarafian,  &  Reynaud,  1997;  Simpson  &  King,  1997).  This  may  be 
associated  with  the  fact  that  ANR  is  most  effective  at  attenuating  noises  with  frequency 
components  below  1000  Hz,  a  region  which  is  not  particularly  important  for  syllable 
discrimination,  and  hence  not  likely  to  be  critical  for  message  intelligibility  (French  & 
Steinberg,  1947).  Another  possibility  is  that  ANR  will  actually  reduce  the  effects  of 
spatial  audio,  since  it  involves  modification  of  the  sound  wave  before  it  reaches  the 
listener’s  eardrum. 

In  order  to  explore  the  effects  of  ANR  on  spatial  audio,  a  speech  intelligibility 
evaluation  similar  to  the  field  investigations  mentioned  above  was  conducted  aboard  an 
AWACS  aircraft  during  a  training  mission  (Bolia,  2003).  In  this  study,  personnel 
listened  to  two  simultaneous  talkers  presented,  in  the  spatial  intercom  condition,  on  both 
sides  of  their  heads.  Although  no  overall  effect  of  ANR  was  discovered,  it  was  found  to 
improve  intelligibility  in  the  most  challenging  noise  condition,  when  the  target  talker  on 
the  right  side  of  the  head  was  masked  by  highly  directional  aircraft  noise.  In  this  case, 
ANR  significantly  improved  speech  intelligibility.  One  possible  explanation  for  this 
phenomenon  is  that  ANR  reduced  the  ambient  noise  level  -  and  hence  enhanced  the 
signal-to-noise  ratio  -  in  the  region  in  which  ITDs  operate,  thus  allowing  the  operator  to 
use  spatial  cues  not  otherwise  available  in  the  noise  environment.  This  hypothesis, 
however,  has  not  endured  subsequent  investigation,  and  an  adequate  explanation  has  yet 
to  be  found. 

While  these  results  have  not  been  fully  explained,  they  do  raise  an  interesting 
possibility.  If  the  noise  field  in  which  operators  are  located  is  known  or  can  be  measured, 
it  should  be  possible  to  design  an  adaptive  spatial  intercom,  so  that  the  locations  of 
talkers  change  in  response  to  operator  location  on  the  platform  or  changes  in  the  noise 
environment  (Bolia,  Nelson,  &  Morley,  2002;  Haas,  Nelson,  Repperger,  Bolia,  & 
Zacharias,  2001).  While  this  presents  an  intriguing  possibility,  designers  should  take  care 
that  real-time  intercom  adaptation  not  result  in  loss  of  situation  awareness  or  reduced 
intelligibility  due  to  the  cost  associated  with  overt  shifts  of  spatial  attention. 


Future  Directions.  The  research  findings  presented  thus  far  provide  compelling 
empirical  evidence  of  the  benefits  of  advanced  interface  technologies  to  enhance 
communication  effectiveness,  reduce  operator  workload,  and  support  the  establishment, 


adaptation,  and  maintenance  of  situation  awareness.  As  noted  above,  these  conclusions 
are  the  result  of  a  converging  methodological  approach  involving  highly-controlled 
laboratory  research,  operator-in-the-loop  simulation  experiments,  field  observations  and 
evaluations,  and  subjective  feedback  from  subject  matter  experts.  Accordingly,  these 
interface  technologies  should  be  considered  in  the  overall  system  design,  especially  in 
noisy  and  communication-intense  applications  domains  such  as  air  battle  management.  It 
is  important  to  point  out,  however,  that  this  review  is  somewhat  limited  in  that  its  primary 
focus  has  been  on  the  effects  of  interface  technology  on  verbal  communication. 
Although  verbal  communication  will  continue  to  be  essential  in  BMC2  application 
domains,  the  networked,  massively  data-linked  architecture  will  demand  additional 
modes  of  communication  and  collaboration.  Collectively,  interfaces  technologies  to 
support  these  modes  are  known  as  collaborative  interface  technologies,  and  comprise 
tools  that  enable  synchronous  and  asynchronous  forms  of  communication.  Current 
capabilities  include  email,  instant  messaging  and  chat,  tele-conferencing,  video 
conferencing,  file  and  application  sharing,  collaborative  workspaces,  data  mining  and 
autonomous  agents,  decision  support  and  automation,  content,  knowledge,  and  workflow 
management  systems,  and  interactive  data  visualization  technologies.  One  the  primary 
near-term  challenges  will  be  to  understand  the  capacity  of  these  technologies  to  provide 
heightened  shared  situation  awareness  and  real-time  collaboration  in  network-centric 
coalition  operations.  Along  these  lines,  some  preliminary  work  has  been  conducted  by 
Nelson,  Bolia,  Vidulich,  and  Langhorne  (2004),  which  characterized  some  of  the  barriers 
to  effective  communication  among  coalition  operators.  However,  a  survey  conducted  by 
these  researchers  also  indicated  that  the  perceived  utility  of  collaborative  technologies  to 
support  infonnation  sharing  and  collaboration  was  substantial.  Accordingly,  research 
programs  systematically  addressing  the  evaluation  of  the  technologies  are  warranted. 
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The  Domain 


2 


The  Problem 


ft 


Noise 

—  Relatively  diffuse  jet 
engine  noise  coupled 
with  highly  directional 
noise  due  to 
machinery 

—  Long  duration  flights 
can  lead  to  hearing 
loss 

—  Loud  external  noise 
environments  leads  to 
increased  volume  of 
comm  channels, 
which  leads  to  more 
hearing  loss 


The  Problem 


*  Communications 


—  Important 
communications  made 
difficult  to  hear  by  both 
noise  environment  and 
other  comms 

—  Exacerbated  by  hearing 
loss 


—  Multiple  simultaneous 
communications  from 
sources  on  and  off  the  jet 
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The  Solution? 


•  Active  Noise  Reduction  (ANR)  has  the  potential 
to  reduce  hearing  loss  and  improve  speech 
intelligibility 

•  Spatial  Audio  technology  has  been  demonstrated 
to  improve  intelligibility  in  multi-channel 
communications  environments 
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Active  Noise  Reduction 


•  Uses  microphone  to  measure  external  noise  field 

•  Creates  a  noise  source  with  the  same  frequency 
spectrum  but  reversed  phase 

•  Adding  the  phase-reversed  sound  to  the  original 
sound  cancels  the  noise 

•  Works  well  only  for  low  frequencies 
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Active  Noise  Reduction 


•  AFRL  researchers 
measured  the  AWACS 
noise  environment 

•  Determined  that  ANR 
headsets  could  improve 
communication 
effectiveness  and  reduce 
hearing  loss  for  AWACS 
operators 

•  Problems 

—  Long  duration 
-  Net-4 


Active  Noise  Reduction 


9  ANR  Headset  Evaluation 

-  AFRL  asked  by  ESC/AW  to 
evaluate  candidate  ANR 
headsets  for  acquisition 

—  10  vendors  offered  headsets 
for  a  “fly-off” 

•  Combat  training  sorties 
flown  out  of  Tinker  AFB,  OK 

•  Crew  members  randomly 
assigned  to  one  of  the  ANR 
headsets 

•  Post-flight  questionnaires 
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Active  Noise  Reduction 


*  ANR  Headset  Evaluation  (continued) 

—  Objective  Measures 

•  Noise  Attenuation  Performance 

•  Headset  Clamp  Strength 

•  Weight 

•  Battery  Pack  Weight 

•  Battery  Configuration 

—  Subjective  Measures 

•  Fit 

•  Comfort 

•  Usability 

•  Audio  Quality 

•  Subjective  Hearing  Protection 

•  Fatigue 
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Active  Noise  Reduction 


*  Second  evaluation  phase 

—  1st_  and  2nd-place  headset  vendors  provided 
additional  headsets  which  were  worn  by  randomly 
selected  operators  during  normal  duty  for  four 
weeks 

—  Survey  assessment  tool  indicated 

9  Bose  headset  considered  a  substantial  improvement 
relative  to  current  headset 

•  2nd-place  headset  more  prone  to  comfort  and  usability 
issues 

*  Use  of  ANR  reduced  perceived  Net-4  time 

—  Bose  headset  selected  for  acquisition,  and  is 
currently  in  use  in  the  AWACS  fleet 
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Spatial  Audio  Displays 


•  In  natural  environments,  listeners  make  use  of  spatial  cues 
to  more  efficiently  segregate  speech  channels 

—  Enhanced  Speech  Intelligibility 
—  Reduced  Workload 
—  Improved  Situation  Awareness 

•  Spatial  Audio  Displays  are  designed  to  simulate  naturally 
occurring  spatial  cues  and  present  them  to  listeners  via 
headphones 

•  What  are  the  cues  that  allow  humans  to  localize  sound? 

—  Interaural  Time  Differences  (ITDs) 

—  Interaural  Level  Differences  (ILDs) 

—  Head-Related  Transfer  Functions  (HRTFs) 
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Spatial  Audio  Displays 


•  ITDs 


—  Sounds  arrive  at  the  ear  closest 
to  the  source  about  0-800  jus 
before  they  arrive  at  the  ear 
furthest  away,  depending  on  the 
angle  of  incidence 

—  The  brain  can  detect  these 
differences  and  uses  them  to 
determine  the  location  of  a  sound 
source 

—  Works  for  low-frequency  sounds 
(up  to  1500  or  2000  Hz) 


12 


Spatial  Audio  Displays 


•  ILDs 

—  Sounds  are  louder  at  the  ear 
closest  to  the  source  than  at 
the  ear  furthest  away,  due  to 
head  shadow 

—  The  brain  can  detect  these 
differences  and  uses  them  to 
determine  the  location  of  a 
sound  source 

—  Works  for  high-frequency 
sources  (above  3000  Hz) 
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Spatial  Audio  Displays 


•  HRTFs 


—  By  the  time  sound  reaches 
the  eardrum,  it  has  been 
modified  by  interactions  with 
the  head,  torso,  and  outer 
ears 

—  The  magnitude  of  these 
modifications  varies  as  a 
function  of  frequency 


Sound  source  at  +90°  Azimuth,  0°  Elevation 


—  The  brain  uses  these  cues  to 
determine  the  front/back  and 
up/down  components  of  a 
sound  source’s  location 

—  Highly  individualized 

—  Contains  ITD  and  ILD 
information 


Spatial  Audio  Displays 


Schematic  of  a  two-channel  spatial  audio  display  generator 
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\*«^Task:  Coordinate  Response  Measure  (CRM) 


•  Developed  at  AFRL  in  1980  as  a  basic  task  relevant 
to  military  communications 

•  256  phrases  per  talker 

•  “Ready  call  sign  go  to  color-number  now” 

—  8  call  signs  (Arrow,  Baron,  Charlie,  Eagle, 
Hopper,  Laker,  Ringo,  Tiger) 

—  4  colors  (Red,  Blue,  White,  Green) 

—  8  numbers  (1-8) 

•  Task:  Select  the  color-number  combination  spoken 
by  the  talker  who  spoke  your  assigned  call  sign 
(selective  attention) 
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Task:  Coordinate  Response  Measure  (CRM) 
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Spatial  Audio 
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Optimized  Configuration 
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Studies  at  Tinker  AFB 


Studies  at  Tinker  AFB 


□  Standard  Comm 

□  Spatial  Comm 


Mental  Physical  Temporal  Performance  Effort  Frustration 

Demand  Demand  Demand 


NASA  TLX  Subscale 
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Studies  at  Tinker  AFB 


"How  would  you  rate  your  satisfaction  with  the  spatial  audio  intercom  compared  with  the 

intercom  you  use  operationally?" 


Rating 
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Studies  Aboard  AWACS 


An  AWACS  operator  participating  in  the  experiment 


Studies  Aboard  AWACS 
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Operational  Use 
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Operational  Use 


•  “With  the  use  of  the  volume  controls  and  the  various 
bucket  positions  I  had  total  SA  on  the  fight.  I  could  tell 
the  difference  between  my  aircraft  and  the  adversaries 
and  was  fully  aware  when  the  pilots  were  talking  on 
the  common  safety  frequency.  This  is  the  first  time  in 
my  16  year  career  that  I  have  fully  heard  and 
comprehended  a  multi-frequency  mission.  ” 

•  “Increases  SA  as  to  what  radio  (frequency)  someone 
is  talking  on.  Cuts  down  on  guessing  what  radio  a  TX 
came  across.  ” 

•  “I  think  the  system/capability  is  very  useful  for  certain 
missions.  As  an  SD  listening  to  5  or  6  different 
frequencies,  it  makes  a  big  difference  helping  you 
allocate  SA  to  listen  to  a  particular  freq.  ” 
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Operational  Use 


*  ANR  Headsets 

-  Currently  used  by  552nd  ACW  (USAF  AWACS) 
—  Being  considered  for  other  BMC2  platforms 

*  Spatial  Comm 

—  Currently  used  at  USAF  Weapons  School 
—  Requirement  for  Block  40/45  AWACS  upgrade 
—  Being  considered  by  JSTARS,  E-2C 
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Questions 
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